Multilingual machine translation models can benefit from synergy between different language pairs, but also suffer from interference. While there is a growing number of sophisticated methods that aim to eliminate interference, our understanding of interference as a phenomenon is still limited. This work identifies the main factors that contribute to interference in multilingual machine translation. Through systematic experimentation, we find that interference (or synergy) are primarily determined by model size, data size, and the proportion of each language pair within the total dataset. We observe that substantial interference occurs mainly when the model is very small with respect to the available training data, and that using standard transformer configurations with less than one billion parameters largely alleviates interference and promotes synergy. Moreover, we show that tuning the sampling temperature to control the proportion of each language pair in the data is key to balancing the amount of interference between low and high resource language pairs effectively, and can lead to superior performance overall.
translated by 谷歌翻译
在全球范围内消除语言障碍的目标的驱动下,机器翻译已巩固自己是当今人工智能研究的关键重点。但是,这样的努力围绕着一小部分语言结合在一起,留下了绝大多数低资源的语言。在确保安全,高质量的结果的同时,在牢记道德考虑的同时,打破200个语言障碍需要什么?没有留下的语言,我们首先通过与母语人士的探索性访谈来解决对低资源语言翻译支持的必要性来应对这一挑战。然后,我们创建了旨在缩小低资源和高资源语言之间的性能差距的数据集和模型。更具体地说,我们开发了一种有条件的计算模型,基于专家的稀疏混合物,该模型经过针对针对低资源语言量身定制的新颖有效的数据挖掘技术培训的。我们提出了多次建筑和培训改进,以抵消数千个任务的培训。至关重要的是,我们使用人类翻译的基准,Flores-200评估了40,000多种不同的翻译方向的性能,并将人类评估与新型毒性基准相结合,涵盖Flores-200的所有语言,以评估翻译安全性。我们的模型相对于先前的最新技术,实现了44%BLEU的改善,为实现通用翻译系统奠定了重要的基础。最后,我们开源此工作中描述的所有贡献,可在https://github.com/facebookresearch/fairseq/tree/nllb上访问。
translated by 谷歌翻译
最先进的愿景和愿景和语言模型依靠大规模的Visio-linguisting预借鉴,以获得各种下游任务的良好性能。通常,这种模型通常是跨模态(对比)或多模态(具有早期融合)但不是两者;它们通常只针对特定的方式或任务。有希望的方向将是使用单一整体普遍模型,作为“基础”,目标是一次性的所有方式 - 真正的视觉和语言基础模型应该擅长视力任务,语言任务和交叉和多数模态视觉和语言任务。我们将Flava介绍在这样的模型中,并在跨越这些目标模式的广泛的35个任务上展示令人印象深刻的性能。
translated by 谷歌翻译
Arbitrary pattern formation (\textsc{Apf}) is well studied problem in swarm robotics. The problem has been considered in two different settings so far; one is in plane and another is in infinite grid. This work deals the problem in infinite rectangular grid setting. The previous works in literature dealing with \textsc{Apf} problem in infinite grid had a fundamental issue. These deterministic algorithms use a lot space of the grid to solve the problem mainly because of maintaining asymmetry of the configuration or to avoid collision. These solution techniques can not be useful if there is a space constrain in the application field. In this work, we consider luminous robots (with one light that can take two colors) in order to avoid symmetry, but we carefully designed a deterministic algorithm which solves the \textsc{Apf} problem using minimal required space in the grid. The robots are autonomous, identical, anonymous and they operate in Look-Compute-Move cycles under a fully asynchronous scheduler. The \textsc{Apf} algorithm proposed in [WALCOM'2019] by Bose et al. can be modified using luminous robots so that it uses minimal space but that algorithm is not move optimal. The algorithm proposed in this paper not only uses minimal space but also asymptotically move optimal. The algorithm proposed in this work is designed for infinite rectangular grid but it can be easily modified to work in a finite grid as well.
translated by 谷歌翻译
The size of an individual cell type, such as a red blood cell, does not vary much among humans. We use this knowledge as a prior for classifying and detecting cells in images with only a few ground truth bounding box annotations, while most of the cells are annotated with points. This setting leads to weakly semi-supervised learning. We propose replacing points with either stochastic (ST) boxes or bounding box predictions during the training process. The proposed "mean-IOU" ST box maximizes the overlap with all the boxes belonging to the sample space with a class-specific approximated prior probability distribution of bounding boxes. Our method trains with both box- and point-labelled images in conjunction, unlike the existing methods, which train first with box- and then point-labelled images. In the most challenging setting, when only 5% images are box-labelled, quantitative experiments on a urine dataset show that our one-stage method outperforms two-stage methods by 5.56 mAP. Furthermore, we suggest an approach that partially answers "how many box-labelled annotations are necessary?" before training a machine learning model.
translated by 谷歌翻译
This paper considers the problem of data-driven prediction of partially observed systems using a recurrent neural network. While neural network based dynamic predictors perform well with full-state training data, prediction with partial observation during training phase poses a significant challenge. Here a predictor for partial observations is developed using an echo-state network (ESN) and time delay embedding of the partially observed state. The proposed method is theoretically justified with Taken's embedding theorem and strong observability of a nonlinear system. The efficacy of the proposed method is demonstrated on three systems: two synthetic datasets from chaotic dynamical systems and a set of real-time traffic data.
translated by 谷歌翻译
We present SpeechMatrix, a large-scale multilingual corpus of speech-to-speech translations mined from real speech of European Parliament recordings. It contains speech alignments in 136 language pairs with a total of 418 thousand hours of speech. To evaluate the quality of this parallel speech, we train bilingual speech-to-speech translation models on mined data only and establish extensive baseline results on EuroParl-ST, VoxPopuli and FLEURS test sets. Enabled by the multilinguality of SpeechMatrix, we also explore multilingual speech-to-speech translation, a topic which was addressed by few other works. We also demonstrate that model pre-training and sparse scaling using Mixture-of-Experts bring large gains to translation performance. The mined data and models are freely available.
translated by 谷歌翻译
Network-based analyses of dynamical systems have become increasingly popular in climate science. Here we address network construction from a statistical perspective and highlight the often ignored fact that the calculated correlation values are only empirical estimates. To measure spurious behaviour as deviation from a ground truth network, we simulate time-dependent isotropic random fields on the sphere and apply common network construction techniques. We find several ways in which the uncertainty stemming from the estimation procedure has major impact on network characteristics. When the data has locally coherent correlation structure, spurious link bundle teleconnections and spurious high-degree clusters have to be expected. Anisotropic estimation variance can also induce severe biases into empirical networks. We validate our findings with ERA5 reanalysis data. Moreover we explain why commonly applied resampling procedures are inappropriate for significance evaluation and propose a statistically more meaningful ensemble construction framework. By communicating which difficulties arise in estimation from scarce data and by presenting which design decisions increase robustness, we hope to contribute to more reliable climate network construction in the future.
translated by 谷歌翻译
肾脏是人体的重要器官。它保持体内平衡并通过尿液去除有害物质。肾细胞癌(RCC)是肾癌最常见的形式。大约90%的肾脏癌归因于RCC。最有害的RCC类型是清晰的细胞肾细胞癌(CCRCC),占所有RCC病例的80%。需要早期和准确的CCRCC检测,以防止其他器官进一步扩散该疾病。在本文中,进行了详细的实验,以确定可以在不同阶段诊断CCRCC的重要特征。 CCRCC数据集从癌症基因组图集(TCGA)获得。考虑了从8种流行特征选择方法获得的特征顺序的新型相互信息和集合的特征排名方法。通过使用2个不同的分类器(ANN和SVM)获得的总体分类精度来评估所提出方法的性能。实验结果表明,所提出的特征排名方法能够获得更高的精度(分别使用SVM和NN分别使用SVM和NN),与现有工作相比,使用SVM和NN分别使用SVM和NN进行分类。还要注意的是,在现有TNM系统(由AJCC和UICC提出的)提到的3个区分特征中,我们提出的方法能够选择其中两个(肿瘤的大小,转移状态)作为顶部 - 大多数。这确立了我们提出的方法的功效。
translated by 谷歌翻译
在本文中,我们通过合并的回归策略来预测条件生存函数。我们将弱的学习者视为不同的随机生存树。我们建议在右审查设置中最大化和解以找到最佳参数。我们探索两种方法,一种通常的生存眼镜蛇和基于一致性指数的新型加权预测指标。我们提出的配方使用两种不同的规范,例如Max-Norm和Frobenius Norm,从测试数据集中的查询点找到了一组邻近性预测。我们通过三个不同的现实数据集实现来说明我们的算法。
translated by 谷歌翻译